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METHOD, APPARATUS AND SYSTEM FOR BLACK SEGMENT 
DETECTION 

CROSS REFERENCE TO RELATED APPLICATIONS 

This application claims benefit under 35 U.S.C. § 1 19 of United States Provisional 
Patent Application Serial Number 60/237,776, entitled Method, Apparatus And System 
For Black Segment Detection, which was filed on October 1, 2000. 

FIELD OF THE INVENTION 

The present invention relates generally to image processing, and more particularly 
to black segment detection in digital film processing. 

BACKGROUND OF THE INVENTION 

Digital images are becoming more popular with consumers, especially as people 
become accustomed to sending and receiving information over the Internet. Waiting 
more than a few seconds for an image can be excruciating to many people. Photographs 
from an analog camera, however, may take days to develop. The fastest way to receive 
analog photographs at the present time is by taking exposed film to one of the many 
popular "One-Hour" developers. 

The developers process the film and return negatives and prints on photo paper 
to the customer. This process is typically completed within one hour. Prints, however, 
are relatively useless if the customer wants to transmit the photographs through a network 
or manipulate them with a computer. Since computers perform operations digitally, a 
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photograph must be digitized before it can be loaded into a computer. Digitizing a 
photograph requires a scanner, which increases costs. Also, scanning an analog image 
involves an inherent loss of resolution. 

Applied Science Fiction, Inc. has developed digital film processing to directly 
5 digitize an image from exposed photographic film. This process eliminates the expense 

and resolution losses associated with digitizing photographs using a scanner. The images 
are stored directly into a digital format that the consumer may instantly transmit through 
a network or view on a computer screen. 

During digital film processing the digital image is normalized using a value for 

10 black and a value for white. Typically the value for black is determined from pixels from 

within the area of the exposed leader of the negative that have low gray levels. 
Conventional methods for normalization require the intervention of a trained 
photographic laboratory technician to scan each negative to locate the exposed leader and 
visually identify an area of dark pixels spread over a relatively large area. This approach 

15 results in inconsistency in selecting values used for normalization and also inefficiency 

due to the long time required for hand selection. 

In addition, the leader section of a film is almost always overexposed to light at 
the time the film is loaded into a camera. As a result, this section of the film is always 
considerably darker than its immediate surrounding areas. Selecting the darkest point on 

20 the leader does not consistently provide an optimum black level because the darkest point 

is not usually representative of a black portion of the images that need to be normalized. 
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What is desirable is finding an area of very dark pixels spread over a large, contiguous 
area vv'ithin the area of the leader of the film. 

As a result, there is a need for a method, apparatus and system for detecting black 
segments, which are large, contiguous areas of very dark pixels, and thus determining 
optimum black values quickly and consistently from exposed film leaders. 
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SUMMARY OF THE INVENTION 

The present invention provides a method, apparatus and system for detecting 
black segments, which are large, contiguous areas of very dark pixels, and thus 
determining optimum black values quickly and consistently from exposed film leaders. 
More specifically, the present invention provides a method, apparatus and system for 
selecting a dark area within a film or digital image. The gray level of the selected dark 
area is then used to determine the optimum black level to normalize the images being 
processed. 

More specifically, the present invention provides a method for selecting a dark 
area in a digital image having a number of columns, each column having a number of 
pixels and each pixel having a gray level. The method includes the steps of identifying 
one or more areas having a substantially uniform gray level within the digital image, 
determining an effective darkness value for each identified area, and selecting the dark 
area corresponding to the identified area having the highest effective darkness value. 

In addition, the present invention provides a method for selecting a dark area 
within a film by creating a digital image by scanning the film and cropping the digital 
image. The cropped digital image contains a number of columns, each column having 
a number of pixels, each pixel having a gray level. The present invention then samples 
the columns, determines an average gray level for each sampled column, determines a 
partial derivative of the average gray level for the sampled columns, and filters the partial 
derivative. Thereafter, one or more areas are identified using the filtered partial 
derivative, and an effective darkness value is determined for each identified area. A dark 
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area is then selected which corresponds to the identified area having the highest effective 
darkness value. 

The present invention also provides a computer program embodied on a computer 
readable medium for selecting a dark area in a digital image having a number of columns, 
each column having a number of pixels and each pixel having a gray level. The computer 
program includes a code segment for identifying one or more areas having a substantially 
uniform gray level within the digital image, a code segment for determining an effective 
darkness value for each identified area, and a code segment for selecting the dark area 
corresponding to the identified area having the highest effective darkness value. 

In addition, the present invention provides a computer program embodied on a 
computer readable medium for selecting a dark area within a film. The computer 
program includes a code segment for creating a digital image by scanning the film and 
a code segment for cropping the digital image. The cropped digital image includes a 
number of columns, each column having a number of pixels, each pixel having a gray 
level. The computer program also includes a code segment for sampling the columns, 
a code segment for determining an average gray level for each sampled column, a code 
segment for determining a partial derivative of the average gray level for the sampled 
columns, a code segment for filtering the partial derivative, a code segment for 
identifying one or more areas using the filtered partial derivative, a code segment for 
determining an effective darkness value for each identified area, and a code segment for 
selecting a dark area corresponding to the identified area having the highest effective 
darkness value. 
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The present invention also provides an imaging system having at least one light 
source operable to illuminate a film, at least one optical sensor operable to detect light 
from the film, and an image processor coupled to the optical sensors. The image 
processor creates a digital image from the light detected by the optical sensors, identifies 
one or more areas having a substantially uniform gray level within the digital image, 
determines an effective darkness value for each identified area, selects a dark area 
corresponding to the identified area having the highest effective darkness value, and 
normalizes the digital image using the dark area. 

Other features and advantages of the present invention shall be apparent to those 
of ordinary skill in the art upon reference to the following detailed description taken in 
conjunction with the accompanying drawings. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

The above and further advantages of the invention may be better understood by 
referring to the following description in conjunction with the accompanying drawings in 
which corresponding numerals in the different figures refer to corresponding parts in 
5 which: 

FIGURE 1 is a perspective view of a scanning device in accordance with a digital 
film processing system; 

FIGURE 2 is an illustration of a duplex film processing system in accordance 
with a digital film processing system; 
10 FIGURE 3 depicts a black segment detection process, which selects a dark area 

in a digital image, in accordance with one embodiment of the present invention; 

FIGURE 4 depicts a black segment detection process, which selects a dark area 
in a digital image, in accordance with one embodiment of the present invention; and 

FIGURES 5A, 5B, 5C, 5D, 5E and 5F depict an example of the black segment 
15 detection process, which selects a dark area in a digital image, in accordance with one 

embodiment of the present invention. 
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DETAILED DESCRIPTION 

While the making and using of various embodiments of the present invention are 
discussed herein in terms of a digital film processing system using photographic media, 
it should be appreciated that the present invention provides many applicable inventive 
concepts which can be embodied in a wide variety of specific contexts. For example, the 
present invention can be used in any image processing system where an optimum value 
for black is needed. The specific embodiments discussed herein are merely illustrative 
of specific ways to make and use the invention and do not delimit the scope of the 
invention. 

The present invention provides a method, apparatus and system for detecting 
black segments, which are large, contiguous areas of very dark pixels, and thus 
determining optimum black values quickly and consistently from exposed film leaders. 
More specifically, the present invention provides a method, apparatus and system for 
selecting a dark area within a film or digital image. The gray level of the selected dark 
area is then used and the optimum black level to normalize the images being processed. 

The term "film" is used hereinafter to refer to any unrestricted length of material. 
The film may or may not have aligned and evenly spaced perforations, which are 
hereinafter referred to as "sprocket holes." Camera or motion picture film is, of course, 
a primary example, but the present invention is not to be construed to be limited to a film 
for still camera or even motion picture film. The film may be a strip of material for other 
purposes as well. 
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A digital film scanning apparatus is shown in HGURE 1 . The scanning apparatus 
100 operates by converting electromagnetic radiation from an image to an electronic 
(digital) representation of the image. The image being scanned is typically embodied in 
a physical form, such as on a photographic media, i.e., film, although other media may 
be used. In general, the electromagnetic radiation used to convert the image into a 
digitized representation is preferably infrared light. The scanning apparatus 100 generally 
includes a number of optic sensors 102. The optic sensors 102 measure the intensity of 
electromagnetic energy passing through or reflected by the film 112. The source of 
electromagnetic energy is typically a light source 110 which illuminates the film 112 
containing the scene image 104 and the reference patch image 108 to be scanned. 
Radiation from the source 1 10 may be diffused or directed by additional optics such as 
filters (not shown) and one or more lenses 106 positioned near the sensors 102 and the 
film 112 in order to illuminate the images 104 and 108 more uniformly. Furthermore, 
more than one source may be used. Source 1 10 is positioned on the side of the film 112 
opposite the optic sensors 102. This placement results in sensors 102 detecting radiation 
emitted from source 110 as it passes through the image 104 on the film 112. Another 
light source 111 is shown placed on the same side of the film 112 as the sensors 102. 
When source 111 is activated, sensors 102 detect radiation reflected by the images 104 
and 108. This process of using two sources positioned on opposite sides of the film 112 
being scanned is described in more detail below in conjunction with FIGURE 2. 

The optic sensors 102 are generally geometrically positioned in arrays such that 
the electromagnetic energy striking each optical sensor 102 corresponds to a distinct 
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location 114 in the images 104 and 108. Accordingly, each distinct location 1 14 in the 
scene image 104 corresponds to a distinct location, referred to as a picture element, or 
"pixel" for short, in the scanned, or digitized image 105. The image 104 on film 1 12 are 
usually sequentially moved, or scanned, across the optical sensors 102. The optical 
sensors 102 are typically housed in a circuit package 1 16 that is electrically connected, 
such as by cable 1 18, to supporting electronics for computer data storage and processing, 
shown together as computer 120. Computer 120 may then process the digitized image 
105. Alternatively, computer 120 may be replaced with a microprocessor and cable 118 
replaced with an electrical circuit connection. 

Optical sensors 102 may be manufactured from different materials and by 
different processes to detect electromagnetic radiation in varying parts and bandwidths 
of the electromagnetic spectrum. The optical sensor 102 may include a photodetector 
(not expressly shown) that produces an electrical signal proportional to the intensity of 
electromagnetic energy striking the photodetector. Accordingly, the photodetector 
measures the intensity of electromagnetic radiation attenuated by the image 104 on film 
112. 

Turning now to FIGURE 2, a conventional color film 112 is depicted. As 
previously described, the present invention uses duplex film scanning that refers to using 
a front source 216 and a back source 218 to scan the film 112 with reflected radiation 222 
from the front 226 and reflected radiation 224 from the back 228 of the film 1 12 and by 
transmitted radiation 230 and 240 that passes through all layers of the film 1 12. While 
the sources 216, 218 are generally monochromatic and preferable infrared. The 
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respective scans, referred to herein as front, back, front-through and back-through, are 
further described below. 

In FIGURE 2, separate color levels are viewable within the film 112 during 
development of the red layer 242, green layer 244 and blue layer 246. Over a clear film 
bases 232 are three layers 242, 244, 246 sensitive separately to red, green and blue light, 
respectively. These layers are not physically the colors; rather, they are sensitive to these 
colors. In conventional color film development, the blue sensitive layer 246 would 
eventually develop a yellow dye, the green sensitive layer 244 a magenta dye, and the red 
sensitive layer 242 a cyan dye. 

During development, layers 242, 244, and 246 are opalescent. Dark silver grains 
234 developing in the top layer 246, the blue source layer, are visible from the front 226 
of the film, and slightly visible from the back 228 because of the bulk of the opalescent 
emulsion. Similarly, grains 236 in the bottom layer 242, the red sensitive layer, are 
visible from the back 228 by reflected radiation 224, but are much less visible from the 
front 226. Grains 238 in the middle layer 244, the green sensitive layer, are only slightly 
visible to reflected radiation 222, 224 from the front 226 or the back 228. However, they 
are visible along with those in the other layers by transmitted radiation 230 and 240. By 
sensing radiation reflected from the front 226 and the back 228 as well as radiation 
transmitted through the film 112 from both the front 226 and back 228 of the film 1 12, 
each pixel for the film 1 12 yields four measured values, one from each scan, that may be 
mathematically processed in a variety of ways to produce the initial three colors, red, 
green and blue, closest to the original scene. 
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The front signal records the radiation 222 reflected from the illumination source 
216 in front he the film 112. The set of front signals for an image is called the front 
channel. The front channel principally, but not entirely, records the attenuation in the 
radiation from the source 216 due to the silver metal particles 234 in the top-most layer 
246, which is the blue recording layer. There is also some attenuation of the front 
channel due to silver metal particles 236, 238 in the red and green layers 242, 244. 

The back signal records the radiation 224 reflected from the illumination source 
218 in back of the film 112. The set of back signals for an image is called the back 
channel. The back channel principally, but not entirely, records the attenuation in the 
radiation from the source 218 due to the silver metal particles 236 in the bottom-most 
layer 242, which is the red recording layer. Additionally, there is some attenuation of the 
back channel due to silver metal particles 234, 238 in the blue and green layers 246, 244. 

The front-through signal records the radiation 230 that is transmitted through the 
film 112 from the illumination source 218 in back of the film 112. The set of front- 
through signals for an image is called the front-through channel. Likewise, the back- 
through signal records the radiation 240 that is transmitted through the film 112 from the 
source 216 in front of the film 112. The set of back-through signals for an image is called 
the back-through channel. Both through channels record essentially the same image 
information since they both record the attenuation of the radiation 230, 240 due to the 
silver metal particles 234, 236, 238 in all three red, green, and blue recording layers 242, 
244,246 of the film 112. 
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Several image processing steps are required to convert the illumination source 
radiation information for each channel to the red, green, and blue values similar to those 
produced by conventional scanners for each spot on the film 112. These steps are 
required because the silver metal particles 234, 236, 238 that form during the 
development process are not spectrally unique in each of the film layers 242, 244, 246. 
While the present invention is described in context of a digital film processing system, 
it is understood that conventional scanners could also be used. 

Now referring to FIGURE 3, a black segment detection process 300 which selects 
a dark area in a digital image in accordance with one embodiment of the present 
invention is shown. Note that the digital image has a number of columns, each column 
has a number of pixels and each pixel has a gray level. In addition, only one channel of 
data (see FIGURE 2) is necessary to selected the dark area. The process 300 starts in 
block 302 and identifies one or more areas having a substantially uniform gray level 
within the digital image in block 304. An effective darkness value is then determined for 
each identified area in block 306. A dark area is then selected that corresponds to the 
identified area having the highest effective darkness value in block 308. The process 
ends in block 310. The gray level of the dark area is then used as the optimum black 
value during normalization of the digital images scanned from the film. 

The identification of one or more areas having a substantially uniform gray level 
within the digital image in block 304 may include several steps, such as determining an 
average gray level for each column, determining a partial derivative of the average gray 
level for the columns, and identifying one or more areas using the partial derivative. The 
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partial derivative of the average gray level for the columns can be determined by 
averaging the average gray level of pixels within a column with the average darkness 
level of pixels within a succeeding column. The results of the partial derivative can be 
made more robust by first sampling the columns of the digital image and then taking the 
partial derivative of the average gray level of the sampled columns. Filtering the partial 
derivative also improves the results. One way of filtering the partial derivative is by 
using a threshold function. 

In block 306, the effective darkness value for each identified area may be 
determined by weighting a gray level of the identified area with a size value of the 
identified area. The gray level of each identified area may be an average gray level of 
the pixels within the identified area, or a median gray level of the pixels within the 
identified area. The size value of each identified area may be the number of pixels within 
the identified area, or the number of columns within the identified area. 

Referring now to FIGURE 4, a black segment detection process 400 which selects 
a dark area in a digital image in accordance with one embodiment of the present 
invention is shown. Process 400 starts in block 402 and a digital image is created by 
scanning film in block 404. The digital image is then cropped in block 406 to delete a 
specified number of rows of pixels from the top and bottom of the digital image that 
contain the area around the film sprocket holes. The areas around the sprocket holes are 
deleted because these areas contain spurious gray level information which is unrelated 
to finding the optimal black level. In the case of 35mm film, the top and bottom 64 rows 
of pixels are deleted. Next, the columns of the cropped digital image are sampled in 
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block 408. Sampling the columns of the cropped digital image provides several 
advantages over using the complete or high resolution digital image. Sampling improves 
the results of the partial derivative step in block 412. Sampling also reduces the amount 
of memory required and the amount of time required to select the dark area. In the case 
of 35mm film, sampling every eighth column provides very good results. Typically in 
such a case, the dark area will be found within the first 2048 sampled columns of the 
digital image. Other sampling rates may also be used. 

The average gray level of the pixels within each sampled column, //(X), is 
determined in block 410 according to the following formula: 

iv r=o 

The sampled digital image of the film roll is represented hyf(X, Y) and has dimensions 
MxN pixels. X and Y represent the x and y coordinates respectively of each pixel. 

Areas of uniform gray level along the length of the film can be determined by 
taking the partial derivative along the x direction. The partial derivative of the average 
gray level for the sampled columns within the digital image are determined in block 412. 
The general formula for calculating /fX; is given as: 
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In the discrete case, where f(X, Y) is an image of dimensions M x N pixels, f"(X) is 
determined as follows: 



Because parameters required for determining /'(M-l) and/i:M-2) lie outside of the 
sample set for the digital image, the results for/tM-1) and/tM-2) are set to "0". 

To further distinguish the areas having more uniform gray levels, the partial 
derivative/ tX) is filtered using a threshold function in block 414. For each value off XX) 
over an appropriately set threshold value, t, the corresponding value of T{X) is set to "0". 
For each value off XX) equal to or less than the appropriately set threshold value, t, the 
corresponding value of T(X) is set to "1". 



For example, a threshold value t of "4" works well with a sampling rate of "8" so that all 
sharp edges along the length of the film are represented as "O's" in r(X) and all areas of 
nearly uniform gray levels are represented as "1 ' s" in T(X). Each sequence of successive 
"I's" in TiX) represents an area of nearly uniform gray levels and is identified by giving 
each area a separate and unique label in block 416 according to a Connected Component 
Labeling algorithm. (See Gonzalez, et al. Digital Image Processing, Addison-Wesley, 
Reading, Mass., 1992.) Each area of nearly uniform gray level is separated from other 
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such area by elements of the r(X) array having a value of "0", which values of "0" 
represent edges. 

Each identified area is examined to determine the "brightness" {e.g. average gray 
level) of the entire area by calculating either a median or average of the magnitude or 
5 gray levels of all of the pixels that are contained within the identified area in block 418. 

The number of pixels or the number of columns within the identified area are also 
determined in block 418. If v is defined as the brightness of the identified area, wherein 
the brightness is directly proportional to the magnitude of the pixel, 
V = Median (\iiXp)) 



or 

Np xp 



1 0 where Xp represents each pixel belonging to an identified area P and Np is defined as the 

number of pixels in the identified area. Optionally, areas containing less than 300 
columns (for a sample set of 2048 columns) may be ignored for the purposes of selecting 
the black area. 

Because v represents the brightness of an identified area, v must be 
15 complemented to determine the "darkness" of an area for the purpose of finding the 

darkest area covering the largest area. If v is made to vary such that 0 < v <1, the 
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number of equivalent pixels {i.e. the effective darkness value of an area) can be calculated 
according to: 

e(P) = (1 - v)Np 

where eis defined as the effective darkness value of the identified area or the number of 
equivalent pixels in the identified area, and Np is defined as the number of pixels or the 
number of columns in the identified area. The effective darkness value is calculated for 
each identified area in block 420 and the identified area with the highest effective 
darkness value is selected in block 422 to be the black area. Thereafter, the black 
segment detection process ends in block 424 when the location of the black area is 
provided to the normalization routine of the image processing system. The normalization 
routine uses values from the black area to determine the optimum black level for 
processing the digital images. Alternatively, the black segment detection process may 
determined the average gray level of the dark area and provide that data directly to the 
normalization routine. 

FIGURES 5A, 5B, 5C, 5D, 5E and 5F depict an example of the black segment 
detection process, which selects a dark area in a digital image, in accordance with one 
embodiment of the present invention. In HGURE 5 A, the leader portion 500 of film 1 1 2 
(HGURES 1 and 2) is shown having a top sprocket hole area 502, a bottom sprocket hole 
area 504 and several areas 506, 508, 510, 512, 514 and 516. Each area 506, 508, 510, 
5 12, 5 14 and 5 16 has a substantially uniform, but different, gray level associated with it, 
as indicated by the cross hatching. 
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HGURE 5B represents a digital image 520 after the top and bottom sprocket hole 
areas 502 and 504 (FIGURE 5 A) have been cropped. The areas 502 and 504 (FIGURE 
5A) are deleted because these areas contain spurious gray level information which is 
unrelated to finding the optimal black level. In the case of 35mm film, the top and 
5 bottom 64 rows of pixels are deleted. In one embodiment, the digital image 520 is 

represented as a bi-dimensional array of pixels with an 8-bit word representing the 
monochromatic value or gray level of each pixel (0 to 255) and with the x-axis of the 
array aligned along the direction of travel of the film 112 (HGURES 1 and 2). Next, the 
columns of the cropped digital image 520 are sampled. The sampled columns are 
10 indicated by lines 522. Sampling the columns of the cropped digital image 520 provides 

several advantages over using the complete or high resolution digital image. Sampling 
improves the results of the partial derivative step (see FIGURE 5D). Sampling also 
reduces the amount of memory required and the amount of time required to select the 
dark area. In the case of 35mm film, sampling every eighth column provides very good 
15 results. Typically in such a case, the dark area will be found within the first 2048 

sampled columns of the digital image. Other sampling rates may also be used. 

Now referring to HGURE 5C, the average gray level p.(X) of the pixels within 
each sampled column 522 (FIGURE 5B) are determined and represented by line 524. 
Areas of uniform gray level 506, 508, 5 10, 5 12, 5 14 and 5 16 along the length of the film 
20 112 can be determined by taking the partial derivative/TX^ of the average gray level ^I(X) 

for the sampled columns 522 (FIGURE 5B) along the x direction. 
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FIGURE 5D shows the absolute value of the partial derivative \fXX}\oi the 
average gray level for the sampled columns |a(X) within the digital image 520 (HGURE 
5B) and is represented by line 526. To further distinguish the areas 506, 508, 510, 512, 
5 14 and 5 16 having more uniform gray levels, the partial derivative/ "(X), represented by 
line 526, is filtered using a threshold function. As illustrated in FIGURE 5E, for each 
value of/tX), represented by line 526 (FIGURE 5D), over an appropriately set threshold 
value, t, the corresponding value of T{X) is set to "0". For each value of /'(X) equal to 
or less than the appropriately set threshold value, t, the corresponding value of T{X) is set 
to "1". For example, a threshold value t of "4" works well with a sampling rate of "8" 
so that all sharp edges along the length of the film 1 12 are represented as "O's" in T{X) 
and all areas of nearly uniform gray levels are represented as "I's" in r(X). Each 
sequence of successive " 1' s" in T{X) represents an area of nearly uniform gray levels and 
is identified by giving each area a separate and unique label A, B, C, D, E and F 
according to a Connected Component Labeling algorithm. {See Gonzalez, et al.. Digital 
Image Processing, Addison- Wesley, Reading, Mass., 1992.) The areas of nearly uniform 
gray level A, B, C, D, E and F are separated from each other by elements of the T{X) 
array having a value of "0", which values of "0" represent edges. Optionally, areas 
containing less than 300 columns (for a sample set of 2048 columns) may be ignored for 
the purposes of selecting the black area. 

Now referring to HGURE 5F, each identified area A, B, C, D, E and F is 
examined to determine the "brightness" {e.g. average gray level) of the entire area by 
calculating either a median or average of the magnitude or gray levels of all of the pixels 
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that are contained within the identified area A, B, C, D, E and F. The number of pixels 
or the number of columns within the identified area A, B, C, D, E and F are also 
determined. The effective darkness value of each identified area A, B, C, D, E and F is 
determined by weighting the complement of the average gray level for the identified area 
A, B, C, D, E and F by the size value of the identified area A, B, C, D, E and F. The size 
value may be the number of pixels or the number of columns within the identified area 
A, B, C, D, E and F. The identified area with the highest effective darkness value, which 
is identified area E in this example, is then selected to be the black area. Thereafter, the 
black segment detection process ends in block 424 when the location of the black area 
is provided to the normalization routine of the image processing system. The 
normalization routine uses values from the black area to determine the optimum black 
level for processing the digital images. Alternatively, the black segment detection 
process may determined the average gray level of the dark area and provide that data 
directly to the normalization routine. 

The present invention is useful in any film processing application. The invention 
is most advantageous where consistency and speed is desired in processing digital images 
from photographic negatives. Although preferred embodiments of the invention have 
been described in detail, it will be understood by those skilled in the art that various 
modifications can be made therein without departing from the spirit and scope of the 
invention as set forth in the appended claims. 
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